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L~:> . Abstract. A new concept of mutual pressure is introduced for potential functions on both 

04 ' continuous and discrete compound spaces via discrete micro-states of permutations, and its 

f-H , relations with the usual pressure and the mutual information are established. This paper is 

■ a continuation of the paper of Hiai and Petz in Banach Center Publications, Vol. 78. 
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Introduction 



' Entropy and pressure are two basic quantities in statistical physics as well as information 

Ph ' theory, which are in the duality relation via the Legendre transforms of each other. Mutual 

• information is another important entropic quantity in information theory. The aim of this 

, paper is to seek for the mutual version of pressure whose Legendre transform is equal to the 

mutual information. 

The mutual information of two random variables X and Y is defined as the relative entropy 

I{X AY):=S{fi(^x,Y)\\f^x(^fiY), 

I where fJ-(x,Y) is the joint distribution measure of {X, Y) and nx <8> fJ-y is the product of the 

I respective distribution measures of X, Y. This is also expressed as 

I{X AY) = -S{X, Y) + S{X) + S{Y) 

o ■ 

Q"^ I in terms of the Shannon entropy S{-) when X,Y are discrete random variables. When X,Y 

' are continuous variables, the expression holds with the Boltzmann-Gibbs entropy H{-) in 

place of S{-) (as long as H{X) and H{Y) are finite). These definitions and expressions are 
^ ' naturally extended to the case of more than two random variables. 

In the classical (= commutative) probability setting, we developed in the previous paper 
[5] a certain "discretization approach" to the mutual information by using "discrete micro- 
states" of permutations. In this paper we apply the same idea to introduce the notion of 
the "mutual pressure" for (continuous) potential functions on compound phase spaces. We 
consider the n-fold product [—R,R]'^ of the bounded interval [—R,R], which is regarded as 
the phase space for an n-tuple of real bounded random variables. For a real continuous 
function h on [—R, i?]" the usual pressure of h is given by 



P{h) := log / e'^W dx. 



-R,R]"- 

For an n-tuple (/ii, . . . , /i„) of probability measures on [—R, R], we choose an approximating 
sequence (.^i(A^), . . . , ^„(A^)) such that Ci{N) are vectors in [—R, R]^ (having the coordinates 
in increasing order) and S,i{N) — >■ /ij in moments as ^ oo for 1 < i < n. We define the 
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mutual pressure Psym(/i : Mi) • • • ; A*n) of h with respect to (/xi, . . . ,/Xn) to be the Umsup as 
iV — GO of the asymptotic average 



— log 



^ exp(iVK^(/i(ai(6(iV)), . . . ,a„(e„(7V))))) 

(Ti,...,(t„gSjv 



over permutations cjj € S'at, where KAr(/i(xi, . . . , x„)) := ;^ /'(-^ij , • • • , a^nj) for Xj = 

(xji, ■ ■ ■ ,Xijq) £ [— -R, -R]^, 1 < i < n (Definition 2.1). Then the inequality 

n 

P{h) > Psym{h : /Xl, . . . , |Un) + ^ 

1=1 

is shown to hold, and the equality case is characterized in a natural way (Theorem 3.2). 
Moreover, for a probability measure n on [— i?, i?]" with marginal measures on 
[—R, R], the Legendre transform of Psym{h : /ii, . . . , //„) is shown to be equal to the mutual 
information —H{^) + Y^=i as long as H{^i) > — oo for 1 < z < n (Theorem 3.5). 

The same approach can be also applied to the setting of discrete phase spaces, when 
the Shannon entropy takes the place of the Boltzmann-Gibbs entropy. We deal with the 
discrete case in Section 4 separately since the discussions are considerably diflFerent from the 
continuous case due to the difference of entropies. 



1. Preliminaries in the continuous case 

Let R > and n G N be fixed throughout. We denote by Pvoh{[—R, R]"') the set of 
probability measures on the n-fold product [—R,R]'^ (c M"), and by Ck([— i?, i?]") the real 
Banach space of real continuous functions on [—R, R]^ with the sup-norm ||/|| := max{|/(x)| : 
X G [—R,R]"-}. The Boltzmann-Gibbs entropy of a probability measure n on [—R,R]"' is 
defined to be 

H{i^) := — p{x)logp{x) dx 

if /X has the joint density p(x) with respect to the Lebesgue measure dx on R^; otherwise 
H{fi) := — oo. A measure fi G Prob([— i?, i?]"") typically arises as the joint distribution of an 
n-tuple {Xi, . . . , Xn) of real random variables bounded by R (i.e., \Xi\ < R) on a, probability 
space. In this case, we have H{n) = H{Xi, . . . , Xn)- 

For avector x = {xi, . . . ,xn) in R^ we write ||x||i := iV~^ Sj^i kil- The mean value of 
x is given by 

1 ^ 

For each N,m and S > Owe define A^(/x; N, m, S) to be the set of all n-tuples (xi, . . . , x„) 
of Xj = {xii, ... , XiN) G [—R, R]^, i <i <n, such that 

|K7v(Xn •••XjJ -/^(Xji •■■XjJI < S 

for all ii, . . . , ifc G {1, . . . , n} with 1 < k < m, where Xj^ • • • Xj^^ stands for the pointwise 
product, i.e., 

Xji • ■ ■ Xjj, := (Xj^i • • • Xii^i, Xi^2 ■ ■ ■ Xi^2i ■ ■ ■ ) iCiiAf ' ' ' 3^ikN) £ R'^j 
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and 

/^(s^il ■ ■ ■ Xif^ ) . — / X{-^ ■ ■ ■ Xjj, dfl(^Xi , . . . , Xji)- 

Then it is known [T, 5.1.1] that the limit 

1 

exists, where stands for the Lebesgue measure on M.^ , and furthermore we have 



Jim -logXf'iARif,;N,m,6)) 



Hif,) = hm hm 1 log A|"(A,j(//; N, m, 5)). 

In [5] we introduced some kinds of mutual information /sym(/f^) and /sym(/u), and established 
their relations with H{p) as follows. 

Definition 1.1. Let /x € Prob([— i?, i?]") and /ij be the restriction (or the marginal) of /i to 
the ith component [—R, R] of [—R, R]^ for 1 < z < n. Choose and fix a sequence of n-tuples 
E{N) = (ei(iV),...,e«(iV)), N en, of R^-vectors ^,{N) in [-R,R]^ := {{xi,...,xn) G 
[-R,R\^ : xi < ■■■ < xn} such that KN{Ci{N)'') f x^ d^ii{x) as iV ^ oo for all A; E N, 
i.e., ^i{N) fii in moments for 1 < i < n. We call such a sequence H(A^) an approximating 
sequence for (^ui, . . . ,^n)- For G N the action of the symmetric group Sn on is given 
by 

cr(x) := (x<^-i(i), . . . 

for a G S'at and x = [xi, . . . ,xj\^) G M^. For each N,m ^ N and 6 > we define Asym(/U : 
S(A^); A^, m, 5) to be the set of all (cii, . . . , cr„) G such that 

(ai(ei(A^)), . . . , dniUN))) G A^(^; iV, m, 6). 

We define 

/sym(/i) := - lim limsup^log7f"(Asym(/U : H(iV);iV,m,5)), 

m^oo,(5\0 iv^oo 

where 75^ is the uniform probability measure on Sjy, and define also IsymifJ-) by replacing 
lim sup by lim inf. This definitions of /sym(/u) and Isym(/u) are independent of the choice of 
an approximating sequence H(A^) for (/xi, . . . ,//„) ([5, Lemma 1.5]). 

Theorem 1.2. ([5, Theorem 1.6]) For every // G Prob([— i?, i?]") marginals ni,..., 
^in ePvohii- R,R]), 

n n 
H{n) = -/sym(/^) + '^H{fJ,i) = -Tsym(/U) + y^-g(/^»)- 
i=l i=l 

The pressure of /i G C]k([— ii, i?]") is given by 



P(^) := log / e'^W dx. 



It is well known that the pressure function P{h) for h G Cir([— i?, i?]") and the (minus) 
Boltzmann-Gibbs entropy —H{ji) for ^ G Prob([— i?, i?]") are in the duality relation in the 
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sense that they are the Legendre transforms of each other. That is, 

H{ii)=mi{-n{h)+P{h) ■.heCu{[-R,Rr)}, /xGProb([-i?,i?]"), (1.1) 

P{h) = max{/i(/i) + H{fi) : fi G Prob([-i?, i?]")}, h G Cr{[-R, i?]"). 

Furthermore, for every h G Cir([— i?, i?]") the Gibbs probability measure fi^ associated with h 
is given by 

dfihix) := — e^^'^Ux with Zf, := I e'^W dx 
which is characterized by the variational equality 

that is, fih is a unique maxiniizer of fi G Prob([— i?, -R]") i— > /u(/i) + H{fi). 

2. Mutual pressure and its Legendre transform 

In the setting of continuous compound spaces described in Section 1, we introduce the 
mutual version of pressure for continuous potential functions, and consider its Legendre 
transform that is a version of the mutual information. 

Definition 2.1. Let G Prob([— i?, i?]) be given and choose an approximating 

sequence E{N) = (^i(7V), . . . , ^„(iV)) of ^i(iV) G [-R,R]^ for (/ui, . . . , ^u^) as in Definition 
II. 1[ For each h G Ck([— i?, i?]") and x,j = {xn, . . . , xin) G [—R, R]^ , I < i < n, define 

/l(xi, . . . ,X„) := (/l(xii, . . . ,X„i),/l(xi2, . . . ,Xn2), • • ■ , h{xiN , ■ ■ ■ , Xun)) G (2.1) 

and hence 

1 ^ 

KAr(/i(xi, . . . ,x„)) := -^'^K^lj, ■ ■ ■ ,Xnj)- (2.2) 

J=l 

For each h G Ck([— -R, R]^) we define the mutual pressure of /i with respect to (^i, . . . , fin) to 
be 



sym 

= hmsupl log / exp{NKM{h{ai{^i{N)), . . . , a„(e„(iV))))) djf;^{ai, . . . ,a„) 



AT 



lim sup — log 



(iV!)" 



^ exp(iVK^(/i(ai(ei(iV)), . . . ,a„(e„(iV))))) 



The above definition is justified by the following: 

Lemma 2.2. Psym(^ : fJ-i, ■ ■ ■ , l^n) is independent of the choice of an approximating sequence 
E{N) for (/ii, . . . ,/i„). 

Proof. Let E'{N) = {^[(N), . . . , Cn(-^)) be another approximating sequence for (/ii, . . . , /i„). 
We write Psym(/'' : E) and Psym(^ : E') for Psym(^ : /^i, • • • , fJ-n) defined in Definition 12.11 with 
E{N) and E'{N), respectively. Since Psym(^ : E) and Psym(^ : E') are continuous in h in the 
norm (see Proposition 12.31 (3) below), it suffices to prove that Psymip '■ E) = Psym(p '■ E') for 
any real polynomial p of n variables xi, . . . , Xn- Since ^i{N),^^{N) G [— i?, R]<, for any e > 
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there exists a 6 > such that, for every iV G N, if \\£,i{N) - < (5 for alH = 1, . . . , n, 

then 

\'^n{p{cti{UN)), cTniUN)))) - i^NipicTiieiiN)), . . . , a„(C(iV)))) | < e 

for ah (ai, . . . , C7„) G S]^. Thanks to [6l Lemma 4.3] (also [H 4.3.4]), there exists an TVq S N 
such that if > A'^o then ||^j(A^) — '^K^)!!! < 5 for ah i = 1, . . . , n. Hence we have for every 



— log 



0"l,...,(T„£S'jv 



— log 



(71,...,(T,jG5jv 



< e. 



This implies that |Psym(p : — -fsym(p : ^01 ^ ^- Since e > is arbitrary, the desired 
conclusion follows. □ 

The following are basic properties of Psym(^ : /^i; • • • > fJ-n), whose proofs are straightforward. 

Proposition 2.3. Let ^i, . . . G Pvoh{[—R,R]). 

(1) When n=l, Psym{h : /xi) = for all h G C]R([-i?, i?]). 

(2) Psyui{h ■ /Ui, . . . , fin) is a convex and increasing function on Cir([— i?, i?]"). 

(3) \Psym{h : ^1, . . . - Psym{h' : ^1 , . . .,Hn)\ < \\h - h'W for all h,h' G Cr{[- R, R]"^) . 

(4) //I < m < n, G Cr{[-R, R]""), /i^^) g Cr{[-R, R]''-"') and h{xi,...,Xn) : = 

: ^m) ~^ h^'^'^ (^Xm+lj ■ ■ ■ : Xn) , then 
Psym {h ■ fJ-l , • • • , /^n) ^ -fsym {h^ • 1^1: ■ ■ ■ i l-^m) ~l~ -fsym {h^ • ; • • • ) /^n) • 

Definition 2.4. Let /i G Prob([— i?, i?]") with marginals /ii, . . . € Prob([— i?, i?]). Define 

IsymifJ-) ■■= SUp{/i(/l) - Psym(/l : /il, • • • , /i„) : /i G CM([-i?, i?]")}, 

that is, Xsym(^) is the Legendre transform of Psym(^ : ^i, • • • Furthermore, we say that 

fi is mutually equilibrium associated with h G Ck([— ii, i?]") if the variational equality 

^sym(/^) = - Psymih : . . . , jln) 

holds. 

The next proposition says that Psym{h ■ /ii, . . . , is the converse Legendre transform of 

-^sym(/^)' 

Proposition 2.5. For ewery /i G Cir([— i?, i?]") and /ii, . . . G Prob([— i?, i?]), 

Psym(/i : /ii, . . . ,/in) = max{^(/i) - 2sym(/u) : G Prob^^,..,,^„([-P, i?]")}, 

where Prob^^^,,,_^„ ([— i2, i?]") is i/ie set o/ all n G Prob([— i?, i?]") whose restriction to the 
ith component of [— i?, P]"' is for 1 < i < n. Hence there exists a mutually equilibrium 
probability measure associated with h whose marginals are ni, . . . ,iJ,n- 
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Proof. One can consider Prob([— i?, i?]") as a closed convex subset of the dual (real) Banach 
space Cr{[-R,R]'')* of Cr{[-R, R]'^). Let F : Ck{[-R, R]"")* (-00, +00] be the conjugate 
(or the Legendre transform) of Psym(^ : fJ-i, ■ ■ ■ ^fJ-n), i-e., 

F(V') :=sup{V(/i)-Psym(/i:Mi,...,/U„) :/i€C7K([-i?,i?]")} 

for ip G CR([-i?,i?]")*. We then prove that 

(F{fi)=I,yM ifMeProb^,,..„^„([-i?,i?n, 

|F(V') = +cx) ifVGCR([-i?,i?]")*\Prob^i,...,A.„([-^,^]")- 

The first equality is just the definition of Zsym(^). The second follows from the following 
three claims. 

(a) If ip{h) < for some h € Cr{[-R,R]'') with h > 0, then F{ip) = +00. In fact, for 
a < we have Psym{cth : fii, . . . , Hn) < -Psym(0 : fJ-i, ■ ■ ■ , fJ^n) = by Proposition 12.31 (2) so 
that 

ip{ah) - Psym(a/i : Ml, • • • ,/^n) > aV'(^) — ^ +00 

as a — — CX3. 

(b) If ipil) 7^ 1, then F{7p) = +00. In fact, since Psym(al : Ml, • • • ,Mn) = « for a G M, it 
follows that 

Tp{ah) - Psym(al : Ml, • • • ,Mn) = a(V'(l) - 1) — > +00 

as a — > +00 or —00 accordingly as ^(1) < 1 or ^jJ{l) > 1. 

(c) Assume that G Proh{[-R, R]") but /i Prob^i,...,^„ ([-i?, i?]"). Then there exists 
an / G C^{[—R,R]) such that m(/''*'') > Mi(/) for some 1 < i < n, where /^*-*(x) := /(xj) for 
X = (xi, . . . , x„) G [— i?, i?]". Since 

-Psym(a/^*'' : Ml, • • • ,Mn) = hm af{Ci{N)) = amif) 

for Q G M, it follows that 

/i(a/«) - Psym(a/(*^ : /ii, ...,//„) = a(M(/^'^) - Mi(/)) ^ +00 

as a — +00. 

Hence (j2.3p is proved. Since Psym(^ : Mi,---,Mn) is a convex continuous function on 
Cm.{[—R,R]^) by Proposition 12.31 the duality theorem for conjugate functions implies that 

Psymih : fii,...,^in)= sup{V(/i) - F(V') : V € Cu{[-R, RD*} 

= sup{n{h) -Xsym(M) : M e Prob^i,...,^„([-i?,i?]")}. 

Since Prob^^^...^^,^([— i?, i?]") is weakly* compact and 2"sym(^) is weakly* lower semicon- 
tinuous on Prob^^^...^^^ ([— i?, i?]"), the above latter supremum is attained by some fi G 
PToh,,_,„{[-R,Rr). □ 

Proposition 2.6. The function Pgymih : /ii, . . . , //„) zs jointly continuous on Cr([— i?, ii]") x 
(Prob([— i?, i?]))" with respect to the norm topology on Cm.{[—R,R]"') and the weak* topology 
on Proh{[-R, R]). 
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Proof. Let h,h' G Cr{[-R,R]'^) and e Fioh{[-R, R]), I <i <n. For any e > choose 

a real polynomial p of n variables xi, . . . , x„ such that \\p — h\\ < e. We have 

|Psym(/l ■ ■ ■ ■ ,tJ'n) - Psyia{h' : /i'l, • • • 

^ |-fsym(^ : • • • ,/Un) - -Psym(P ■ ■ ■ ■ ,/Un)| 

+ |-Psym(P : ^1, . . . , /i„) - Psym(P : , • • • , /i^) | 
+ : ^'l, . . . , /i^) - Psym(/i' : /i'l, . . . , 

< + \\P - h'W + |Psym(P : /il, . . . - Psym(P : /^i , • • • ,Atn)| 

< 2e+ ||/i - h'W + |Psym(p : ■ ^'l,---,/^n)l 

by Proposition 12.31 (3). Recall that the weak* topology on Prob([— P, P]) is metrizable with 
the metric z/') := X]fc^i(2P)^'^ — where u{x'') := / x'' dv{x). It suffices to 

show that there exists a 5 > such that if p{fj-i, < 5 ioi 1 < i < n, then 

|Psym(P : . . . , ^ln) - Psym(p : ^'l, • • • ,/^^)| < 

One can choose a (5i > such that, for every € N, if Xj,x^ S [— P, P]< and ||xj — x^jji < 5i 
for 1 < i < n, then 

l«^Af(p(o"l(xi), . . . ,cr„(x„))) - KAr(p(cri(xi), . . . ,cr„(x^)))| < £ 

for all (o"!, . . . , cj„) S S'^. Thanks to [6, Lemma 4.3] one can choose an m G N and a 
(52 > such that, for every iV G N, if x,x' G [-P, P]< and \k,n{^^) - /tAr(x"')| < 82 for all 
k = 1, . . . ,m, then ||x — x'||i < (5i. Then choose a ^3 > such that if z^, z^' G Prob([— P, P]) 
and p{v,v') < 6-^, then — < 62/2 for all /c = l,...,m. Now assume that 

G Prob([-P,P]) and < ^3 for 1 < i < n. Let E{N) = {^i{N), . . . ,Cn{N)) and 

E'{N) = {S,'i{N), . . . , C'ni^)) be approximating sequences for (/^i, . . . , and (/i'^^, . . . , fi'^), 
respectively, with ^i(iV), ^^(iV) g [-P,P]^. There exists an A^o e N such that if > iVo 
then for 1 < i < n we have 

Mm)') - ^^Nim)')\ 

< iKNim)") - ^^^ix'')\ + \p^ix'') " /i^X^')! + IpK^") " '^n(^^(^^)')l < ^2 

for all A: = 1, ... ,m so that - C'iiN)\\i < 5i. Hence if N > Nq then we have 

|'^iv(p(^Ti(ei(iV)), ■ ■ • , (TniUN)))) - ^N{p{c7i{ei{N)), . . . ,a„(C(iV))))I < e 

for all ((Ti, . . . , cr„) G S]^. This implies that \Psym{p : pi, ■ ■ ■ , Pn) - Psym(p : /^i, . . . , < e, 
as required. □ 

Corollary 2.7. The function ZgymifJ') "is weakly* lower semicontinuous on Prob([— P, P]"). 

Proof Let fi and //*^'^\ /c G N, be in Prob([— P, P]") such that /i*^'^) p weakly*. Let fii and 
IJ'l^\ 1 < i < n, be the marginals of fi and /x*-'^-*, respectively. Since fi'l'^^ pi weakly* as 
A; — > 00 for 1 < i < n. Proposition 12.61 implies that for every h G Ck([— P, P]") 

^(/i) -P,ym{h:pu...,Pn)= lim (/i) - Psyn,(/i : p!i\ ...,p^^^)} 

k^oo 

<liminfT,y„,(;u(^)) 

fc— >oo 

so that Xsym(^) < liminffc^oo2rsym(^'''^''), as required. □ 
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3. Relations of Psym(^) with P{h) and of Jsym(^) with H{fi) 

First let us recall the Sanov large deviation in the form suitable for our purpose. Let 
ho G Ck([— i?, i?]) and hq be the Gibbs probability measure associated with ho, i.e., 

(i;^o(x) := ^e'^«(^)dx with Zho := [ e^^'^^'Ux. 
Zho J[-R,R] 

Consider the infinite product probability space ([— ii, ^Uq"^) and i.i.d. (independent 
and identically distributed) random variables xi,X2,--- consisting of coordinate variables 
of [— i?, The Sanov theorem (see [3l 6.2.10]) says that the empirical measure (random 

probability measure) 

Sxi -\ 1- Sxj^ 

N 

satisfies the large deviation principle in the scale with the good rate function S{fj, \\ //q) 
for fi G Prob([— i?, i?]), where S{fi \\ fio) denotes the relative entropy (or the KuUback-Leibler 
divergence) of fi with respect to fiQ. That is, 

limsup 1 log^^r f ^^1±— ±^ G < - m^{S{^, \\ ^o) : ^ G F}, 
liniinf 1 log /x^ (^ ^^^^■■^ + ^^- € g) > - inf {5(m || ^o) : ^ G G} 

for every closed subset F and every open subset G of Prob([— i?, R]) in the weak* topology. As 
remarked in [H p. 211], it then follows (based on the Borel-Cantelli lemma) that the empirical 
measure (Sx^ + • • • + Sxff)/N converges to fiQ in the weak* topology almost surely. In the next 
lemma we state some consequences of the above large deviation, which will play a crucial role 
in our later discussions. 

Lemma 3.1. Let /iq and /Uq be as above. Then: 

(a) For every m G N and 5 > 0, 

hm /x^^(AR(/io;iV,m,5)) = 1. 

N—KX) 

(b) // /ii G Prob([— i?, R]) and ni ^ /xq, then there exist an m G N and a 6 > such that 

limsup i-log/xf^(A^(A.i;iV,m, ,5)) < 0. 

Proof, (a) For each m G N and 6 > set 

G{fio;m,6) := {fi G Pvoh{[-R,R\) : |^(x^') - Mo(^'')| < <5, 1 < A: < m}, 

which is a weak* neighborhood of /no- Note that x = {xi, . . . , xn) G Aji{fio; N,m, 6) is 
equivalent to (^^i + • • • + 5x^)/N G G{no;m, 5). Since {6xi + • • • + 5xj^)/N — > fj.o weakly* in 
the sense of almost sure (with respect to Hq"^) as remarked above, we have 

^^ri^Ri^^o■, N, m, 6)) = (^-±:^^±^ G G(^o; m, 5)) 1 

as ^ oo. 



A NEW APPROACH TO MUTUAL INFORMATION. II 9 

(b) Let /Lii G Prob([— i?, R]) with /ii ^ hq. One can find an m G N and a 5 > so that the 
weak* closed subset 

F{ni;m,6) := € Prob([-i?, i?]) : - < S, 1 < k < m} 

does not contain /iq. The large deviation principle implies that 

limsup ^ log fif^{AR{ni;N, m, S)) 

< limsupllog/xf^f^^li--±^ G F{t^v,m,6)] 

< - mi{S{iJ, II Ho): HE F(/xi; m, S)} < 0, 

because S'(/x || /xq) is weakly* lower semicontinuous and so attains the minimum (> 0) on a 
weakly* compact subset m, (5). □ 

The next theorem gives an exact relation between Psym(^) and P{h). 

Theorem 3.2. For every h G Cr([— i?, R]"-) and every /ii, . . . , /i„ G Prob([— i?, R]), 

n 

P{h) > Psymih : ^il,...,^in) + J2 ^('"^)- (^-1) 

i=l 

Moreover the following conditions are equivalent: 

(i) P{h) = Psymih : /ii, . . . ,/i„) + Y17=i ^iP-i)^ 

(ii) Hi, . . . , fin o,re the marginals of the Gibbs measure associated with h; 

(iii) for each i = 1, . . . ,n, jii is the Gibbs measure associated with hi G C]r([— i?, R]) defined 
by 



hi{x) := log / ^h{xi,...,xi-ux,xi+i,...,xn) fj^^^... dxi_idx 

J[-R,R]"-^ 

forxe [-R,R]. 



Proof. Consider the Gibbs probability measure := ^e^^^^ c/x associated with h so that 
P{h) = \ogZh. Let N,m and 6 > 0. Then it is straightforward to see that 

(n \ » n 

r\AR{fXi;N,m,6)] = / exp(A^/«jv(/i(xi, . . . , x„))) TTc/xi, 

r=l / ^nr=i AK(/.i;iV,m,5) f4 

where ]Xi=i ^nit^u "t-; ^) iii the left-hand side is regarded subset of (M")^ by the cor- 
respondence (xi, . . . , x„) ^ {{xii)i^i, (xi2)i"=i, • • • , ixiN)2=i} for Xj = (xii, . . . , XiN). Hence 
we have 

= (iV!)" 



X ^ exp(iVK7v(/i((ji(xi), . . . ,cr„(x„)))) JJdxj. (3.2) 

Cri,...,(Tn6Sjv *=1 
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Let H(A^) = (^i(A^), . . . , S^niN)) be an approximating sequence for (/ii, . . . , /i^). For any e > 
there exists a real polynomial p of variables xi, . . . ,Xn such that — < e. Then there exist 
an m G N, a J > and an iVo G N such that, for every > Nq, if Xj G AR{fii; N, m, 5) D M< 
for 1 < i < n, then we have 

|KAr(p(cTi(xi), . . . ,CJn(x„)) - Kat (p((Ti (A^) ) , . . . , Cr„ (^n (A^) ))) | < £ 

SO that 

|KAr(/l(cri(xi), . . . ,cr„(x„)) - KAr(/l(cri (iV)), . . . , cr„(Cn (A^)))) | < 3e 
for all (Ti, cr„ G Sat. Hence by ()3.2p we obtain 

> ^''"""ij^ E exp{NKN{h{a,{^,{N)), . . . , a„(e„(iV))))) 
(Ti,...,(7„e5jv 

n 

]jA;v(AK(/ii;iV,m,d)) (3.3) 



n 

X 

1=1 



and 



^ "'"^'n^ E exp(iVKjv(/i(ai(ei(A^)),...,cT„(en(iV))))) 



(Ti,...,(7nG5]V 

n 

x[]A^(AR(/Xi;Ar,m,5)). (3.4) 

j=i 

It follows from (j3.3p that 
1 

iV 



P(/i) = -logZ,^ 



> -3e + ^log 



5^ exp(ArKjv(/i(cTi(ei(Ar)), • . . , a„(en(A^))))) 



+ E ^ AAr(A/j(^i; A^, m, (5)). 

This yields 



N 

i=l 



1 

P(/i) > -3e + Psym(/i : /"I, • • • ,M„) + lim — log AAr(Aij(/ii; A^, m, 5)) 

^ — ' N—^oo i\ 
i=l 

thanks to the existence of the limits in the last term. Letting m — > oo and 5 \ gives 

n 

P{h) > -3e + Psym{h : /ii, . . . , l^n) + ^H{fii), 

i=l 

which implies inequality ()3.ip since e > is arbitrary. 
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Next let us prove the equivalence of conditions (i)-(iii). For 1 < i < n let fj,h,i be the ith 
marginal of fi^. Since 



. . . dx,_idXi+i ■■■dXn]dx 



= _L e^»(^') dx, (3.5) 
Zh 

we notice that is the Gibbs measure associated with hi for 1 < i < n. Hence (ii) 44> (iii) 
follows. To prove (ii) (i), assume that ^Uj = fih,i for all i = 1, . . . , n. Then Lemma 13.11 (a) 
gives 

hm ^f^({(xi,...,x„) € ([-i?,i?H" :x, G Anifif, N,m, 6)}) 
= hm ^^f''iAR{^,i■N,m,5)) = l. 

iV— >oo 

Therefore, 

hm ^^f^lf\An{^^^■,N,m,6)] 



\i=l 



= lim n{(xi,...,x„) :x, G AR(^,;iV,m,5)} = 1. 

Hence it follows from (13.41) that 



P{h) < 3e + Psym{h : /ii, . . . ,/i„) + ^H{ni), 

i=l 

which implies equality in (i). 

Conversely, assume (i). Since ()3.3p implies that 



Pih) + lim sup ^ log /.f ^ ( f[ Ani^^^■, N, m, 6) 



\i=l 



> -3e + Psym(/i : • • • ,Ain) + ^^^(/Ui), 

i=l 

we have 

hmsupllog^f^f fr A^,(/ii;iV,m,5) 1 > -3e. 

Here we can take m arbitrarily large and 6 > arbitrarily small for any given e > 0. 
Therefore, 



limsup4log/xf^(fTA^(/Xi;iV,m,5) I =0 



for all m G N and all 5 > 0. Since 

fif^ (fl An{^l^; N, m,6)]< A^f,f (A^(^,; N, m, 6)), 



\i=l 

we have 



limsupllog/iff(A^(^,;A^,m,5)) = 
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for all m G N, 5 > and i = 1, . . . , n. Lemma [3. 11 (b) implies that m = ^h,i for alH = 1, . . . , n, 
so (ii) holds. □ 

Remark 3.3. Let H(A^) = (^i(iV), . . . , ^„(A^)) be an approximating sequence for (//i, . . . , ^u^)- 
Let hi, . . . ,hn € Ck([— i?, i?]) and consider hi as an element of Cir([— i?, i?]") depending on 
the ith variable Xj, 1 < i < n, so that {hi + ■ • • + /in)(x) = /ii(xi) + • • • + hn{xn) for 
X = (xi, . . . , Since 

^ ^ exp(iVK;v((/i - (/ii + • • • + /i„))(ai(6(iV)), • • • , cT„(e„,(iV))))) 



exp(iVK^(/i(ai(6(iV)), . . . , cjniUN))))) 



CTl,...,CT,ifcijv 

n 

X J]exp(-iVAijv(/iite(iV)))) 

4 = 1 

and limAT^^oo K,N{hi{£_i{N))) = fii{hi), it follows that 

-Psym(/i - (/ll H h /l„) : /il, . . . + ^-P(/ii) 

i=l 

n 

= Psym{h : fii, . . . ,fin) + '^{-Hi{hi) + P{hi)). 

i=l 

Hence we notice that 



Psym{h : Hi,...,Hn) H{fli 



i=l 

= inf lPsjm{h-{hi^ \-hn) : fii,...,Hn)+y^P{hi)}, 

where /ii + • • • + /i„ is given as above for /ii, . . . , /i„ G Ck([— i?, i?]). In particular, when /Xj is 
the Gibbs measure associated with hi for 1 < i < n, we have 

n n 

Psym(/l : /^l, ■ ■ ■ + ^-H'(^i) = Psym(/i " (/il H h /in) : ■ ■ ■ ,/^n) + ^P(/ii). 

4 = 1 j=l 

Hence, if the equivalent conditions (i)-(iii) of Theorem 13.21 are satisfied, then the equality 

n 

P{h) = Psym{h - {hi + ■ ■ ■ + hr,) : ;Ul,...,//„) +^P(/l,) 

4 = 1 

holds as well for /ii , . . . , /i„ given in (iii) . 

The next lemma is concerned with general relation between Xsym(^) and /sym(^)- 
Lemma 3.4. Tsym(/^) < -/^sym(^) for every /i G Prob([— i?, i?]"). 

Proof. Let /i G Prob([— i?, i?]") and /Ui, . . . , ^u^ be the marginals of fi, and choose an approx- 
imating sequence H(A^) = (,^i(A^), . . . , ^„(A^)) for (^i, . . . , fin)- It suffices to prove that 

-fsym(/W) > /W(P) - -Psym(P : /il , . . . , /in) 
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for all real polynomials p of variables xi, . . . ,Xn- For any e > there exist an m € N and a 
6 > such that, for every S N, if (cii, . . . , cr„) G Asym(/^ : ^(-^); "i, 5) then 

\KN{p{ai{UN)), . . . , (T„(e„(iV)))) - < e 

so that 

eA^(Mp)-^) < exp(iV^^(p(ai(ei(iV)), . . . , a„(en(iV))))) . 

Therefore, 

< 7^ E exp(iVK^(j,(ai(6(iV)),...,a.(en(iV))))) 

(o-i,...,o-„)eAsym(Ai:H(Ar);Af,m,<5) 

^ E exp(iVK^(p(ai(ei(iV)),...,a„(e„(iV))))), 

which implies that 

This gives the desired inequality since e > is arbitrary. □ 

The next theorem gives an exact relation between Zsym{^J') and H{fj,). 
Theorem 3.5. For every /i G Prob([— ii, i?]") with marginals fii, . . . ,/x„ E Fioh{[—R, R]), 



i=l 

Moreover, if H{fii) > — oo for all i = 1, . . . , n, then 

IsymiP-) = -^sym(^) = S{n,fJ,i (g) • • • (g) fin), 

and Xsyin(//) = if and only if fi = fii ® ■ ■ ■ Hn, i.e., the coordinate variables 
independent with respect to fi. 

Proof By ([33]) and Definition [231 for every h G Cr{[-R,R]'^) we have 

n 

-fi{h) + P{h) > -n{h) + Psym{h : Hi,...,fln)+Y^ H{fli) 

1=1 



>-IsyM + Y,H{^I^). (3.6) 

j=l 

Hence by (jl.ip . Lemma 13.41 and Theorem 11.21 we have 

n n 
1=1 1=1 

SO that the first assertion is proved. The second assertion immediately follows from the first 
and Corollary 1.7]. □ 

Proposition 3.6. Let h E CK([-i2, i?]") and fi G Prob([-i?, i?]"). Let be the 

marginals of /i and hi, . . . ,hn be as given in ( Hi ) of Theorem \3.2[ Then the following are 
equivalent: 
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(i) /i is the Gibbs measure associated with h; 

(ii) fi is mutually equilibrium associated with h and Hi is the Gibbs measure associated 
with hi for each i = 1, . . . ,n. 

Proof, (i) =^ (ii). Assume that // is the Gibbs measure associated with h. By (j3.6p and 
Theorem 13.51 

H{f,) = -f,{h)+Pih) 

n 

> -fi{h) + Psym{h : . . . , + ^ H{iJ.i) 

i=l 

n 
i=l 

Moreover, since /Xj is the ith marginal of /x = ;U/j, it follows as in the proof of Theorem 13.21 
(see (|3.5|) ) that fii is the Gibbs measure associated with hi for 1 < i < n. In particular, 
H{ni) > — oo for alH = 1, . . . , n. Hence 

that is, n is mutually equilibrium associated with h. 
(ii) =^ (i). Assume (ii). By Theorems 13.51 and 13. 2^ 

n 
i=l 

n 

= -fJ.{h) + Psym(/l : /il, . . . ,/i„) + ^H{^li) 

i=l 

= -^JL{h)+P{h) 

SO that (i) follows. □ 

4. The discrete case 

In information theory, random variables mostly take values in a discrete set of alphabets 
and the basic quantity is the Shannon entropy rather than the Boltzmann-Gibbs entropy. So 
the discrete versions of the preceding results in Sections 2 and 3 are of even more importance, 
which are presented in this section. 

Let X = {ti, . . . , tfi\ be a finite set of alphabets and consider the n-fold product A"". The 
Shannon entropy of a probability measure ^ G Prob(<Y) is 

•^C/") := -5^/«(i)log;u(t). 
For each sequence x = (xi, . . . , x^) G , the type of x is a probability measure on X given 

by 

z^x(t) := where N^{t) := #{j : xj =t}, te X. 

For each /i G Prob(A') (resp. /x G Prob(r^")) and for each G N and 5 > we denote by 
A(/i; A, 5) the set of all sequences x G X^ (resp. x G (Af")^) such that \v-x_{t) — ^{t)\ < S for 
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all t £ X (resp. t G Af"), that is, A{fj.]N,6) is the set of all (5-typical sequences (with respect 
to n). The Shannon entropy has the following limiting formula: 

S{fi) = lim hm log #A(^; N, 6) (4.1) 

(see [HE] and also [5l §2] for a concise exposition). 

For iV G N let denote the set of all sequences of length N of the form 

X = {ti, . . . ,ti,t2, . . . ,t2, ■ ■ ■ ,td, ■ ■ ■ ,td) 

so that is regarded as the set of all types from . The action of Sn on is similar 
to that on given in Definition II. li 



Definition 4.1. Let fi E Prob(^") and /ij S Prob(^) be the ith marginal of fi for 1 < i < n. 
Choose an approximating sequence H(A^) = (^i(A^), . . . , (,n{N)), € N, for (/ii, . . . , /i„), that 
is, £,i{N) G and f^,(7v)(^) ~* /^i(^) as iV ^ cxo for all t £ X and i = 1, . . . ,n. For each 
G N and 5 > we define Asym(/u : S(A^); A^, 5) to be the set of all (cri, . . . , an) G 5"]^ such 
that 

(ai(ei(A^)), . . . , a„(en(A^))) € A(^; iV, 5). 

We define 

hyM := -lim limsup-log7|;(Asyn,(/i : H(iV);iV,5)) 

and /sym(Ai) by replacing lim sup by lim inf. See [5l Lemma 2.4] for the independence of the 
choice of H(A^) for /sym(Ai) and /sym(/^) as well as their equivalent definitions. 

The two quantities /sym(/u) and /sym(/u) are equal and connected to S{iJ,) as follows. 

Theorem 4.2. ([5, Theorem 2.5]) For every fi G Prob(^%'") with marginals /zi,...,//„ G 
Prob(;f), 

n 

hyraifJ') = 7sym(/^) = -5'(/U) + ^ 5(/ii). 

i=l 

We denote by Cr{X"') the real Banach space of real functions on A'" with the norm 
= max{|/(x)| : x G A""}. 



Definition 4.3. Let //i, . . . , £ Prob(A:') and choose an approximating sequence H(A^) = 
(^i(A^), . . . , ^„(A^)) for (ni, . . . , fin) as given in Definition 14.11 For each h G CM.iX^) and 
Xj G A'-'^, 1 < « < n, define /i(xi, . . . ,x„) and KAr(/i(xi, . . . ,x„)) in the same manner as in 
^TDi and ([22]) so that 

KAr(/i(xi, . . . ,x„)) = /l(t)i^(xi,...,x„)(*) 

for (xi, . . . ,x„) regarded as a sequence in (X^)^ . We define the mutual pressure of h with 
respect to (^i, . . . , /i„,) to be 



lim sup — log 



(Ti,...,(t„G5]v 
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Moreover, for each fi E Prob(Af") with marginals . . . , Hn G Prob(<%') we define 

IsymifJ') ■■= SUp{/i(/l) - Psym(/l : fJ-1, ■ ■ ■ , fJ-n) ■ h & C^i'^"')}, 

and we say that is mutually equilibrium associated with h if the equality 

2^sym(/") = /"(/i) - Psym{h : /il , . . . , /U„) 

holds. 

Then all the results in Section 2 are valid in this discrete setting as well. To see this, it is 
convenient to reduce the discrete case to a special case of the continuous case of Section 2 in 
the following way. Choose d points ti < t2 < • • • < in [—R, R] corresponding to ti,t2, ■ ■ ■ ,td 
in X. For each fx € Prob(A'") with marginals ^i, . . . , we have the corresponding (atomic) 
probability measure fi € Prob([— i?, i?]") given by 

fi-=Y^ /^(x)<5x, 

and similarly /li, . . . , /i„ G Prob([— i?, R\), 1 < i < n. Then the marginals of p, are fii, . . . , fin- 
For each approximating sequence (^i(A^), . . . , ^ri(-^)) for (^ui, . . . , ^n) we have the correspond- 
ing ii{N) £ [-R,R]^, l<i<n. Since 

KNiiiiN)'') = i''^i^{N){t) Yl = x'' dfLi{x) as iV ^ oo 

for all A; G N, it follows that (^i(A^), . . . , ,^„(A^)) is an approximating sequence for (/ii, . . . , fin)- 
For each h G Ck(<^") choose an /i G C^{[-R, Rf) such that /i(x) = /i(x) for all x G Then 
we notice that Psjm{h ■ /Ui, . . . , fin) in Definition 14.31 is equal to Psym(^ : Aii • • • > An) defined 
in Definition 12.11 and that Tsyin(/i) in Definition 14.31 is equal to Tsym(A) defined in Definition 
12.41 Upon these considerations it is rather straightforward to show the discrete versions of 
the results in Section 2. For example, for h,h' G Cm.{'^"') choose h,gG Cm.{[—R,R\"') such 
that h\x"- = h, \\h\\ = \\h\\, g\x^ = h — h' and H^H = \\h — and define h' := h — g. Then 
h'\x^ = h! and \\h — h'\\ = \\h — h'\\. Hence the discrete version of Proposition 12.31 (3) is seen 
as follows: 

|-fsym(^ 

= |-fsym(^ : Al, • • • , An) ~ Psyra{h' : fll, . . . , An)| 

< \\h - h'W = \\h- h'\\. 
Also, the discrete version of Proposition 12.51 is seen as follows: 

Psym {h ■ fl\ ) . . . ) /^n) — Psyraih ■ fl\ ; • • • j An) 

= max{A(/i) -2-sym(A) : A G Prob^,,...,^„ ([-i?, i?]")} 
= max{/i(/i) - Jsym(/u) : /i G Prob^i,...,^„(A'")} 

since Prob^,,...,^„([-i?,i?]") = {A : G Prob^,,...,^^,^'")}. 

Now let us show the discrete version of Theorem l3.2[ Although the proof is essentially same 
as that of Theorem 13.21 some non-trivial modifications are necessary due to the difference 
between the Shannon and Boltzmann-Gibbs entropies. 
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Theorem 4.4. For every h G C^{X'^) and every /xi, . . . , Hn G Prob(<Y), 

n 

P{h)>P,y^{h:fii,...,i^n) + Y,S{fii), (4.2) 

1=1 

and the following conditions are equivalent: 

(i) P{h) = Psym{h : /ii, ...,//„) + X;r=i 'S'(^i); 

(ii) /ii, . . . are t/ie marginals of the Gibbs measure fi^ associated with h given by 

/.;,(x):=^e'^W, xGA-" with Z;, := e^W; (4.3) 

(iii) /or eac/i i = 1, . . . ,n, Hi is the Gibbs measure associated with hi G Cui^) defined by 

hi{x) ■.= l0g ^h(xu...,Xi.^,X,X, + u...,Xn) 

for X ^ X . 

Proof. Let ii^ be the Gibbs measure given in (j4.3p . and let (^i(A^), . . . , ^„,(A^)) be an approx- 
imating sequence for (/ii, . . . , /i„,). For any e > one can choose a (5 > such that for every 
i = 1, . . . , n and every p G Prob(Af), if |p(f)— /Xj(t)| < (5 for all t £ X, then |5(p)— 5(/Lij)| < e/n. 
This means that for each N € N and i = 1, . . . ,n, one has |»S'(z/'x) — 'S'(/ii)| < e/n whenever 
x G A(fii;N,5). Furthermore, when 5 > is small enough, one can find an Nq G N such 
that, for every N > Nq, if Xj G Aim; N, 6) n X^ for I < i < n, then 

|k7v(/i(cti(xi), . . . , (T„(X„)) - K7v(/i(fTl(6(A^)), • • • , f^n(en(A^)))) | < S (4.4) 

for all (cTi, . . . ,(T„) G ^at. 

For each sequence (iVi, . . . , iV^) of integers A^; > with X;f=i = let ^(iVi, ■ ■ ■ , No) 
denote the subgroups of Sn consisting of products of permutations of {1, . . . , A^i}, {A^i + 

1, . . . , A^i + iV2}, {Ni-\ h Nd-i + 1, . . . , iV}, and let Sn/S{Ni, ...,Nd)he the set of 

left cosets of ^(A^i, . . .,Nd). For each x G ;f I" we write for 5(A^x(ti), • • • , A^x(id)). For 

G N it then follows that 

zK"|^nA(/i,;iV,5)^ 

= ^ exp(A^KAr(/i(xi, . . . ,x„))) 

(xi,...,x„)Gnr=i A(M.;Af,5) 

E 

(xi,...,x„)Gnr=i(A(M,;Af,5)nA'|') 

exp(iVKAr(/i((Ji(xi), . . . ,cr„(x„)))), (4.5) 

([(Ti],...,[ct„])G(Sjv/Sxi ,.--.5jv/5'x„) 

where YYi=i A{fii; N , 6) in the left-hand side is regarded as a subset of (X"")^ in the same 
manner as in the beginning of the proof of Theorem \'6.'2\ and [uj] denotes the coset of Sxj 
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containing ai. Moreover we have 

exp(7VKAr(/l(cTi(xi), . . . ,C7n(x„)))) 

(71,...,(T,ig5]V 

/ n d \ 

E i]ll[N^M)l\e^p{NKN{h{ai{^i),...,an{^n)))). 
(ki],...,[a„])G(5jv/5xi,...,Sjv/Sx„) Vi=U=l / 

(4.6) 



For each i = 1, . . . ,n and for any x € , the Stirhng formula impHes that 



l5^1ogiV,(tO!-^logiV! 



1=1 



1 / 1 

- ( ^^x(tO log iVx(tz) - N^iti) + - log N^iti) + 0(1) 

- 1 (^iV log iV - iV + i log iV + 0(1) j 

j;^^logiVx(ti)-logiV + o(l) 
z=i 

— 5'(^'x) + o(l) as — > oo, 



where o(l) as ^ oo is uniform for x G X'^ . Thanks to the above choice of 5 > 0, for every 
(xi, . . . , x„) G nr=i ^il^i'i ^! ^) we have 



exp 



Nl^-J2S{f^,)- 8 + 0(1) j 



Ni-Y,Sifii) + e + o{l) 



i=l 



as TV ^ oo, (4.7) 



where o(l) is uniform for (xi, . . . , x„) G ]Xi=i ^if^ij '^)- 
Combining (ji3]) - (|iT71) yields 



> 



E 



^ X] exp(iVKAr(/l(cri(xi), . . . ,CJ„(x„)))) 



(iV!) 



X exp 



NiY,Si^J.^)-e + oil) 



vi=l 
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and the reverse inequality with +e in place of —e in the last term. By this together with 
([O]) we obtain 



> e 



-2Ne 



{my 



cri...,CT„G5jv 

n 

X 

= 1 



and 



< e 



\i=l 
2Ne 



{my 



exp{NKN{h{ai{Ci{N)), MUN))))) 



(Ti...,(T,ie5jv 

n 

X 

1=1 



ll#{A{fi^;N,5)nX^)-exp 



N[YS{^^^) + o{l) 



for all N > Nq. Furthermore, since 



so that 



A{fii; N, 6) = {a(x) : x G A(^,; N, 6) D , [a] G Sn/S^} 

m. 



#A{^if,N,6) 



Uf=iN.{tiy: 



we have as inequalities in ()4.7p 



N(s{fi,) 



+ o(r 



n 



#(A(^i;iV,5)nA'^) -exp 

< #A(^i; iV, 5) < #(A(^,; N, 6) n X^) ■ exp 
This and (|4.ip imply that 



N[s{^,i) + - + o{l) 



■- <liminfllog#(A(^i;iV,,5)n^^) 

<limsupi-log#(A(/ii;iV,5)n^^) < - 



n 



It follows from 
1 



that 



p{h) = i-\ogz^ 



>-2e + -lo, 



(4.8) 



(4.9) 



as N —>■ oo. 



(4.10) 



^ ^ exp(iVKjv(/i(cTi(ei(iV)), . . . , a„(en(iV))))) 

(Tl,.--,0"n€5]V 



n ^ n 

+ E ]^ log #(A(/ii; iV, 5) n + ^ 5(/ii) + 0(1) as iV 



oo, 



i=l 
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which imphes that 

n 

P{h) > -3e + Psym{h : /xi,...,;U„) + '^S{fii) 

i=l 

thanks to ()4.10p . Hence inequahty (j4.2p follows since e > is arbitrary. 

To prove the equivalence of (i)-(iii), let fj,h,i be the ith marginal of fih- Then it follows 
that lJih,i{x) = Z^^e^^^^^ and so fih,i is the Gibbs measure associated with /ij for 1 < i < n. 
Hence (ii) (iii) follows. Assume (ii), i.e., that fii = fi^^i for all i = 1, . . . , n. Since we have 
limN^oo ^^f^ i^i^J'i', N,6)) = 1 based on the Sanov theorem as in Lemma [3TT] (a). it follows 
that 

Jim /.r(nA(^,,; iV, 5)^=1 
as in the proof of (ii) =^ (i) of Theorem I3.2[ Combining this with (j4.9p and (j4.10p yields 

n 

P{h) < 3e + Psym(/i : //i, . . . , /x„) + ^ 

i=l 

which implies equality in (i). Conversely, assume (i). Then ()4.8p and (I4.10p imply that 




The same reasoning as in the last part of the proof of Theorem 13.21 gives 

limsup4log/xff(A(^,;iV,5)) =0 

for all (5 > and i = 1, . . . ,n. Since we have a result similar to Lemma 13.11 (b) in the present 
discrete situation, it follows that fii = fj,h,i for alH = 1, . . . , n, and so (ii) holds. □ 

The next theorem and proposition are the discrete versions of Theorem [33] and Proposition 
13. 6[ Since their proofs based on Theorems 14.21 and 14.41 are similar to those in Section 3, we 
omit the details. Here note only that Xsym(/^) < hymifJ-) for every fj, € Prob(<%'") can be 
shown similarly to the proof of Lemma [3. 41 or by the same reasoning as given after Definition 
14.31 and that the Legendre transform expression as in (jl.ip 

S{fi) = inf{-^(/i) + P{h) : h G C7m(A'")} 

is valid for every fi S Prob(<%'"). 

Theorem 4.5. For every fi G Prob(<Y") with marginals . . . , fin G Prob(A:'), 

n 

2sym(^) = -^sym(M) = -S{fJ-) + '^S{fli). 

1=1 

Proposition 4.6. Let h G Ck{X^) and fi G Prob(<%'"). Let ^i, . . . , fin be the marginals of fi 
and hi, . . . ,hn be as given in (iii) of Theorem \4.4\ Then the following are equivalent: 

(i) fj, is Gibbs measure associated with h; 

(ii) n is mutually equilibrium associated with h and fii is the Gibbs measure associated 
with hi for each i = 1, . . . ,n. 
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